In the Claims: 

This listing of the claims will replace all previous versions, and listings, of 
the claims in this application. 

Claims 1-30, 32-38, 40-44 and 46-56 were previously pending. 

Claims 8, 32, 35 and 55 have been amended. 

Claims 57-66 have been added. 

Claims 1-30, 32-38, 40-44 and 46-66 are pending. 



a plurality of instructions that, when executed by one or more processors, causes 
the one or more processors to perform acts including: 
receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 



receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback; 

generating a weighting of feature elements based at least in part on the 
feedback; and 




/\ . (Original) One or more computer readable media having stored thereon 
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selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors. 

2. (Original) One or more computer readable media as recited in claim 1, 
wherein the selecting a new set of potentially relevant images comprises using a 
matrix in determining the distance between one of the new plurality of query 
vectors and one of the plurality of feature vectors, and further comprising 
dynamically selecting the matrix based on both a number of images in the set of 
potentially relevant images for which relevance feedback was input and a number 
of feature elements in the one feature vector. 

3. (Original) One or more computer readable media as recited in claim 2, 
wherein the dynamically selecting comprises using a diagonal matrix if the 
number of images in the set of potentially relevant images for which relevance 
feedback was input is less than the number of feature elements in the one feature 
vector, and otherwise using a full matrix. 

4. (Original) One or more computer readable media as recited in claim 2, 
wherein the dynamically selecting comprises: 

if the number of images in the set of potentially relevant images for which 
relevance feedback was input is not less than the number of feature elements in the 
one feature vector, then using one matrix that transforms the query vector and the 
one feature vector to a higher-level feature space and then using another matrix 
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that assigns a weight to each element of the transformed query vector and the 
transformed feature vector; and 

if the number of images in the set of potentially relevant images is less than 
the number of feature elements in the one feature vector, then using a matrix that 
assigns a weight to each element of the query vector and the one feature vector. 

5. (Previously presented) One or more computer readable media as recited 
in claim 2, wherein X represents an image matrix that is generated by stacking N 
feature vectors, each of length K, corresponding to the set of potentially relevant 
images for which relevance feedback was received and resulting in an (N x K) 
matrix, C represents a weighted covariance matrix of X , det( C ) represents the 
matrix determinant of C, and the matrix comprises a full matrix (W*) that is 
generated as follows: 

^* = (det(C))^C" 1 . 

6. (Previously presented) One or more computer readable media as recited 
in claim 2, wherein w tt represents the kk^ element of matrix W , x k represents the 
k xh feature element, c k represents the standard deviation of the sequence of x k 's, 
the matrix comprises a diagonal matrix with each diagonal element (w M ) being 
generated as follows: 

1 
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7. (Original) One or more computer readable media as recited in claim 1, 



wherein N represents the number of images in the set of potentially relevant 
images for which relevance feedback has been received, n n represents the 
relevance of image n in the set of images, n represents a transposition of a vector 
generated by concatenating the individual n n values, and X represents an image 
matrix that is generated by stacking N training vectors corresponding to the set of 
potentially relevant images into a matrix, and wherein each new query vector ( q ) 
of the new plurality of query vectors is generated as follows: 



/ 8. (Currently amended) On e or mor e comput e r r e adabl e m e dia as r e cit e d 
in claim 1 

One or more computer readable media having stored thereon a plurality of 
instructions that, when executed by one or more processors, causes the one or 
more processors to perform acts including: 

receiving an initial image selection; 

generating a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection; 

selecting a set of potentially relevant images based at least in part on 
distances between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images; 




2>. 



n 
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receiving feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generating a new plurality of query vectors based at least in part on the 
feedback; 

generating a weighting of feature elements based at least in part on the 
feedback; and 

selecting a new set of potentially relevant images based at least in part on 
both the weighting of feature elements and distances between the new plurality of 
query vectors and the plurality of feature vectors , wherein f. represents a 
summation, over the images in the set of potentially relevant images, of a product 
of a relevance of the image and a distance between the query vector and the 
feature vector, and wherein the selecting a new set of potentially relevant images 
comprises combining, for each image, a weighted distance between the plurality of 
query vectors and the plurality of feature vectors, and wherein the weight (m.) for 
each of a plurality ( / ) of distances between a query vector and a corresponding 
feature vector is calculated as: 



9. (Original) One or more computer readable media as recited in claim 1, 
wherein the receiving feedback comprises receiving feedback from a user. 

10. (Original) One or more computer readable media as recited in claim 1, 
wherein the low-level features include: a color moments feature, a wavelet based 
texture feature, and a water- fill edge feature. 
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Ml. (Original) A method of selecting between two types of matrixes to be 
used to weight, based on relevance feedback, a plurality of feature elements for 
image retrieval, the method comprising: 

selecting one of the two types of matrixes based on both a number of 
previously retrieved relevant images and a length of a feature vector including the 
plurality of feature elements. 

12. (Original) A method as recited in claim 11, wherein the selecting 
comprises selecting one of the two types of matrixes based on both a number of 
previously retrieved potentially relevant images which were identified by a user as 
being relevant, and the length of the feature vector including the plurality of 
feature elements. 

13. (Original) A method as recited in claim 11, wherein the plurality of 
feature elements are all elements of the same feature. 

14. (Original) A method as recited in claim 11, wherein the selecting 
comprises using a first type of matrix if the number of retrieved relevant images is 
less than the length of the feature vector, and otherwise using a second type of 
matrix. 
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15. (Original) A method as recited in claim 14, wherein the first type of 
matrix comprises a diagonal matrix and wherein the second type of matrix 
comprises a full matrix. 



16. (Original) A method as recited in claim 11, wherein the selecting 
comprises using a first type of matrix if the length of the feature vector exceeds 
the number of retrieved relevant images by at least a threshold amount, and 
otherwise using a second type of matrix. 

17. (Original) A method as recited in claim 16, wherein the first type of 
matrix comprises a full matrix and the second type of matrix comprises a diagonal 
matrix. 

18. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 



/l9. (Previously presented) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or more 
processors, causes the one or more processors to perform acts including: 

comparing, for each of a plurality of images, a plurality of feature elements 
from a query vector to a plurality of feature elements from a feature vector 
corresponding to the image; 
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identifying a number of potentially relevant images based on the 
comparing; 

receiving user feedback regarding relevancy of one or more of the 
potentially relevant images; 

re-comparing, for each of the plurality of images, the plurality of feature 
elements from the query vector to the plurality of feature elements from the 
feature vector, including using a matrix to compare the feature elements and 
dynamically selecting a type of matrix to use based on both the user feedback and 
the number of the plurality of feature elements; 

identifying a new set of potentially relevant images based on the re- 
comparing; and 

presenting the new set of potentially relevant images to the user. 

20. (Original) One or more computer readable media as recited in claim 
19, wherein the re-comparing comprises dynamically selecting the type of matrix 
to use based on both a number of the potentially relevant images for which user 
feedback has been received and the number of the plurality of feature elements. 

21. (Original) One or more computer readable media as recited in claim 
19, wherein the dynamically weighting comprises using a first type of matrix if the 
number of retrieved relevant images is less than the length of the feature vector, 
and otherwise using a second type of matrix. 
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22. (Original) One or more computer readable media as recited in 21, 
wherein the first type of matrix comprises a diagonal matrix and the second type 
of matrix comprises a full matrix. 



23. (Original) A method comprising: 

generating a query vector corresponding to a feature of one image; 
identifying a feature vector corresponding to the feature of another image; 
identifying a number of training samples for which relevance feedback has 
been received; 

if the number of training samples either equals or exceeds a threshold 
amount, then determining a distance between the query vector and the feature 
vector including transforming the query vector and the feature vector to a higher- 
level feature space and then assigning a weight to each element of the transformed - 
query vector and the transformed feature vector; and 

if the number of training samples does not exceed the threshold amount, 
then determining the distance between the query vector and the feature vector 
including assigning a weight to each element of the query vector and the feature 
vector. 

24. (Original) A method as recited in claim 23, wherein the feature vector 
includes a plurality of feature elements and wherein the threshold amount 
comprises the number of feature elements in the feature vector. 
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25. (Original) A method as recited in claim 23, wherein if the number of 

training samples either equals or exceeds the threshold amount, then determining 

-» — > 
the distance ( g ), where P is a mapping matrix, q is the query vector, x is the 

feature vector, and A is a weighting matrix, as: . 
g = (P(q-h T MP(q-*)). 

26. (Original) A method as recited in claim 23, wherein if the number of 
training samples does not exceed the threshold amount, then determining the 
distance (g), where q is the query vector, x is the feature vector, and A is a 

weighting matrix, as: 

— » -» — ► -> 
g = (q-x) T A(q-x). 

27. (Previously presented) A method as recited in claim 23, further 
comprising: 

repeating the generating, identifying of the feature vector, identifying of the 
number of training samples, and the determining for each of a plurality of features; 
and 

identifying how closely the image and the another image match each other 
by combining the distances between the query vectors and the feature vectors for 
the plurality of features. 

28. (Previously presented) A method as recited in claim 27, wherein the 
identifying how closely the image and the another image match each other 
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comprises calculating a weighted summation of each of the individual distances 
for each of the plurality of features. 

29. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 



/ 30. (Original) A system comprising: 

a query vector generator to generate a query vector corresponding to a 
feature of one image; 

a comparator, coupled to the query vector generator, to, 
identify a feature vector corresponding to the feature of another image, 
identify a number of training samples for which relevance feedback has 
been received, 

if the number of training samples either equals or exceeds a threshold 
amount, then to determine a distance between the query vector and the feature 
vector including transforming the query vector and the feature vector to a higher- 
level feature space and then assigning a weight to each element of the transformed 
query vector and the transformed feature vector, and 

if the number of training samples does not exceed the threshold amount, 
then to determine the distance between the query vector and the feature vector 
including assigning a weight to each element of the query vector and the feature 
vector. 



23. 
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31. (Canceled). 



5 



/ 32. (Currently amended) A method comprising: 

for one of a plurality of images and each of a plurality of features, 
generating, based on [[the]] a set of search criteria, a query vector for the 
feature, 

identifying a feature vector, corresponding to the image, for the feature, and 
determining how closely the feature vector matches the query vector; and 
determining how closely the image matches the set of search criteria based 
on how closely, for the plurality of features, the feature vectors match the query 
vectors, wherein generating the query vector comprises generating the query 
vector based at least in part on user relevance feedback regarding how relevant 
images previously displayed to a user were. 

33. (Previously presented) A method as recited in claim 32, wherein 
identifying the feature vector comprises: 

identifying a low-level feature vector corresponding to the feature; and 
mapping the low-level feature vector to a higher level feature space. 

34. (Original) A method as recited in claim 33, wherein the identifying the 
feature vector further comprises incorporating, into the mapping, relevance 
feedback. 
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35. (Currently amended) A method as recited in claim 32, wherein the 
initial search criteria comprises an image. 

36. (Previously presented) A method as recited in claim 32, wherein the 
determining how closely the feature vector matches the query vector comprises 
determining a distance between the feature vector and the query vector, and 
wherein the determining how closely the image matches the set of search criteria 
comprises calculating a weighted summation of each of the individual distances 
between the feature vectors and the query vectors. 

37. (Original) A method as recited in claim 36, wherein the calculating a 
weighted summation comprises calculating the weighted summation based at least 
in part on user relevance feedback regarding how relevant images previously 
displayed to a user were. 



38. (Previously presented) One or more computer readable media 
including a computer program that is executable by a processor to perform the 
method recited in claim 32. 




39. (Canceled). 




40. (Previously presented) One or more computer readable media as 
recited in claim 41, wherein the identifying the plurality of query vectors 
comprises extracting the plurality of query vectors from the image. 
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/41. (Previously presented) One or more computer readable media having 
stored thereon a plurality of instructions that, when executed by one or more 
processors, causes the one or more processors to perform acts including: 

identifying a plurality of query vectors for one image, each query vector 
corresponding to one of a plurality of features; 

identifying a plurality of feature vectors for another image, each feature 
vector corresponding to one of the plurality of features; 

for each feature, determining a distance between the corresponding query 
vector and the corresponding feature vector; and 

combining the distances to generate a value representing an overall distance 
between the one and the another image, wherein the identifying the plurality of 
query vectors comprises generating the plurality of query vectors based at least in 
part on user relevance feedback regarding how relevant images previously 
displayed to a user were. 

42. (Previously presented) One or more computer readable media as 
recited in claim 41, wherein the determining the distance between the 
corresponding query vector and the corresponding feature vector includes 
incorporating, into the determining, user relevance feedback regarding how 
relevant images previously displayed to a user were. 

43. (Previously presented) One or more computer readable media as 
recited in claim 41, wherein the combining the distances comprises calculating a 
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weighted summation of each of the individual distances between the feature 
vectors and the query vectors. 



44. (Original) One or more computer readable media as recited in claim 
43, wherein the calculating a weighted summation comprises calculating the 
weighted summation based at least in part on user relevance feedback regarding 
how relevant images previously displayed to a user were. 



45. (Canceled). 



/46. (Original) A method of generating a query vector to compare to a 
feature vector of another image, the method comprising: 

receiving feedback regarding the relevance of each image of a set of 
images; 

wherein N represents the number of images in the set of images for which 
user relevance feedback has been received, n n represents the relevance of image n 
in the set of images, n represents a transposition of a vector generated by 
concatenating the individual K n values, and X represents an image matrix that is 
generated by stacking N training vectors corresponding to the set of images into a 
matrix; and 

generating a query vector (q) corresponding to one of a plurality of 
features as follows: 





9 = 



N 



5>. 



n=l 
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47. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 



/ 48. (Original) A method of generating a weight to apply to distances 
between query vectors and feature vectors when combining the distances, the 
method comprising: 

receiving feedback regarding the relevance of each image of a set of 
images; 

wherein f. represents a summation, over the images in the set of images, of 
a product of a relevance of the image and a distance between the query vector and 
the feature vector; and 

generating a weight ( u i ) for each of a plurality ( / ) of distances between a 
query vector corresponding to one of a plurality (/) of features and a feature 
vector corresponding to the one of the plurality ( / ) of features as: 



49. (Original) One or more computer readable media including a computer 
program that is executable by a processor to perform the method recited in claim 



46. 





48. 




50. (Original) A system comprising: 
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a client device; 

a collection of a plurality of images; 

an image server, coupled to the client device and the collection of a 
plurality of images, the image server to receive image retrieval requests from the 
client device and to, 

receive an initial image selection from the client device, 

generate a plurality of query vectors by extracting, for each query vector, 
one of a plurality of low-level features from the initial image selection, 

select a set of potentially relevant images based at least in part on distances 
between the plurality of query vectors and a plurality of feature vectors 
corresponding to low-level features of a plurality of images, 

receive feedback regarding the relevance of one or more images of the set 
of potentially relevant images, 

generate a new plurality of query vectors based at least in part on the 
feedback, 

generate a weighting of feature elements based at least in part on the 
feedback, and 

select a new set of potentially relevant images based at least in part on both 
the weighting of feature elements and distances between the new plurality of query 
vectors and the plurality of feature vectors. 



5 1 . (Original) One or more computer readable media as recited in claim 1 , 
wherein the receiving feedback comprises receiving feedback in a range including 
at least Highly Relevant, Relevant, No Opinion, Irrelevant, and Highly Irrelevant. 
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52. (Original) One or more computer readable media as recited in claim 1, 
wherein the receiving feedback comprises receiving feedback via speech 
recognition. 

53. (Original) One or more computer readable media as recited in claim 
19, wherein the receiving user feedback regarding relevancy comprises receiving 
user feedback in a range including at least Highly Relevant, Relevant, No Opinion, 
Irrelevant, and Highly Irrelevant. 

54. (Original) One or more computer readable media as recited in claim 
19, wherein the receiving user feedback comprises receiving user feedback via 
speech recognition. 



/55. (Currently amended) One or more computer readable media including 
a computer program that is executable by a processor to cause the processor to 
perform acts of: 

receiving user feedback regarding the relevance of each image of a set of 
images, the user feedback forming a range including at least Highly Relevant, 
Relevant, No Opinion, Irrelevant, and Highly Irrelevant; 

wherein AT represents the number of images in the set of images for which 
user r e l e vanc e feedback has been received, n n represents the relevance of image n 
in the set of images, n represents a transposition of a vector generated by 
concatenating the individual n n values, and X represents an image matrix that is 
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generated by stacking N training vectors corresponding to the set of images into a 
matrix; and 

— ► 

generating a query vector (q) corresponding to one of a plurality of 
features as follows: 




56. (Original) One or more computer readable media as recited in claim 
55, wherein the receiving user feedback comprises receiving user feedback via 
speech recognition. 
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(New) One or more computer readable media having stored thereon a 
plurality of instructions that, when executed by one or more processors, causes the 
one or more processors to: 



select a set of potentially relevant images based at least in part on distances 
between a plurality of query vectors extracted from an initial image selection and a 
plurality of feature vectors corresponding to low-level features of a plurality of 
images; 

receive feedback regarding the relevance of one or more images of the set 
of potentially relevant images; 

generate a new plurality of query vectors based at least in part on the 
feedback; 

generate a weighting of feature elements based at least in part on the 
feedback; and 

select a new set of potentially relevant images based at least in part on both 
the weighting of feature elements and distances between the new plurality of query 
vectors and the plurality of feature vectors, wherein f. represents a summation, 
over the images in the set of potentially relevant images, of a product of a 
relevance of the image and a distance between the query vector and the feature 
vector, and wherein the selecting a new set of potentially relevant images 
comprises combining, for each image, a weighted distance between the plurality of 
query vectors and the plurality of feature vectors, and wherein the weight ( u i ) for 
each of a plurality ( / ) of distances between a query vector and a corresponding 
feature vector is calculated as: 



lee@hayes f* 509.324.92s6 



23 



Application No. 09/660,53,6 




58. (New) One or more computer readable media as recited in claim 57, 
wherein the plurality of instructions to cause the one or more processors to select 
comprises instructions to cause the one or more processors to use a matrix in 
determining the distance between one of the new plurality of query vectors and 
one of the plurality of feature vectors, and further comprises instructions to cause 
the one or more processors to dynamically select the matrix based on both a 
number of images in the set of potentially relevant images for which relevance 
feedback was input and a number of feature elements in the one feature vector, 
wherein the instructions to dynamically select comprise instructions to cause the 
one or more processors to use a diagonal matrix when the number of images in the 
set of potentially relevant images for which relevance feedback was input is less 
than a number of feature elements in the one feature vector, and otherwise using a 
full matrix. 

59. (New) One or more computer readable media as recited in claim 57, 
wherein the plurality of instructions to cause the one or more processors to select 
comprises instructions to cause the one or more processors to: 

use one matrix that transforms the query vector and the one feature vector 
to a higher-level feature space and then using another matrix that assigns a weight 
to each element of the transformed query vector and the transformed feature vector 
when the number of images in the set of potentially relevant images for which 
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• 



relevance feedback was input is not less than the number of feature elements in the 
one feature vector; and 

use a matrix that assigns a weight to each element of the query vector and 
the one feature vector when the number of images in the set of potentially relevant 
images is less than the number of feature elements in the one feature vector. 



60. (New) One or more computer readable media including computer 
readable instructions executable by one or more processors to cause the one or 
more processors to select between two types of matrixes to be used to weight, 
based on relevance feedback, a plurality of feature elements for image retrieval, 
wherein the instructions are further executable to cause the one or more processors 
to select one of the two types of matrixes based on both a number of previously 
retrieved relevant images and a length of a feature vector including the plurality of 
feature elements. 

61. (New) One or more computer readable media as recited in claim 60, 
wherein instructions executable to cause the one or more processors to select 
comprise instructions executable to cause the one or more processors to select one 
of the two types of matrixes based on both a number of previously retrieved 
potentially relevant images which were identified by a user as being relevant, and 
the length of the feature vector including the plurality of feature elements. 

62. (New) One or more computer readable media as recited in claim 60, 
wherein the plurality of feature elements are all elements of the same feature. 
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63. (New) One or more computer readable media as recited in claim 60, 
wherein instructions executable to cause the one or more processors to select 
comprise instructions executable to cause the one or more processors to use a first 
type of matrix when the number of retrieved relevant images is less than the length 
of the feature vector, and otherwise use a full matrix. 

64. (New) One or more computer readable media as recited in claim 60, 
wherein instructions executable to cause the one or more processors to select 
comprise instructions executable to cause the one or more processors to use a 
diagonal matrix when the number of retrieved relevant images is less than the 
length of the feature vector, and otherwise use another type of matrix. 

65. (New) One or more computer readable media as recited in claim 60, 
wherein instructions executable to cause the one or more processors to select 
comprise instructions executable to cause the one or more processors to use a full 
matrix when the length of the feature vector exceeds the number of retrieved 
relevant images by at least a threshold amount, and otherwise use another type of 
matrix. 

66. (New) One or more computer readable media as recited in claim 60, 
wherein instructions executable to cause the one or more processors to select 
comprise instructions executable to cause the one or more processors to use a full 
matrix when the length of the feature vector exceeds the number of retrieved 
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relevant images by at least a threshold amount, and otherwise use a diagonal 
matrix. 
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